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ABSTRACT 

The regionalization of the University of Washington 
Medical School represents a beginning in addressing the problem of 
maldistribution of physician manpower in the states 
concerned-Washington, Alaska, Montana, Idaho (WAMI) • The WAMI 
experiment uses the faculties and facilities available in the 
universities of these states to teach first year medical students the 
first quarter of basic sciences. This paper addresses itself to: (1) 
describing the philosophy, history, strategies, and activities of the 
WAMI experiment, (2) identifying the questions that need to be 
answered by the evaluation process; and (3) progress of the Program 
and curriculum evaluation to date. (Author) 
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BACKGROUND 



It is stated that there is a serious shortage of physicians in the United 
States. It is also a fact that there are more qualified appliccints for medi- 
cal school rhan there are places available. The ratio approaches two qualified 



applicants for every available place in 1973. Another fact relative to every 
stare is that it is not feasible, or even desireable to build new schools, nor 
is it feasible or desireable to increase significantly the size of existing 
classes at the majority of medical schools. 

Coupled with the need for more doctors in the United States is the need 
for a better distribution of the available physicians. Prom the map you can 
see the relative distribution of doctors in the United States. Many states are 
below the national ratio of 130 doctors per 100,000 population (Figure 5 )• 

The states of Washington, Alaska, Montana, and Idaho are good examples of 
the difficulty we are trying to describe. The land mass of the four states is 
equal to 1/4 the land mass of the nation. The geography is mountain, desert, 
and maritime. The population is approximately 7 million, with concentrations 
of this population in the greater Seattle area. Anchorage, Spokane, Boise, and 
Pocatello. 



Health care delivery in the area ranges from the most sophisticated in the 
v;orld in the major metropolitan area to non-existent health care in remote vil- 
lages. A mere increase in the number of physicians graduating will not guarantee 
better health care delivery to the remote areas of the region. Chronic maldistri- 
bution of physicians as well as the shortage of physicians plagues the region. 
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Alaska has only one half of its fair share of physicians based on its population. 
Idalio has a physician/population ratio of 89/100,000 and Montana's is 100/100,000. 
Washington's physician population is only slightly less than the national average 
but most of those physicians gravitate toward the Puget Sound and Spokane areas. 
This maldistribution of physicians which is seen alsq^in Alaska, Montana and 
Idaho leaves large rurl^l^'eas'^^r^ttis^l^ and with many 

significant health care needs. 

Consider for a moment the facilities for training physicians in the region. 
There is a high quality health science complex in Seattle. Alaska, Montana and 
Idaho do not have medical schools. They do have applicants to medical schools 
but have a lower ratio of acceptance because states with medical schools tend to 
keep the majority of their available spaces for their native sons and daughters. 
Only 8 of the 10,546 students admitted to the 100 medical schools in this country 
in 1969 were Alaskan residents. In the same year 25 Idaho, and 27 Montana resi- 
dents were admitted to medical schools. These figures do not represent an increase 
over the number admitted in 1951 or 1952 in spite of the explosion in undergradu- 
ate enrollments during this period. 

It would appear that students from these states are being discouraged from 
applying to medical school. Because of the current reluctance of ftinding agengies 
to provide money for new buildings or expansion of existing ones to accomodate 
more students, a radical departure from the traditional physical milieu of medi- 
cal education seems necessary. The challenge to medical education is not only 
to increase the nvunber of physicians from states that have fewer applicants but 
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also to achieve a better distribution of physicians in areas that now suffer 
chronic shortage, and all this without devoting significant resources to building 
new facilities. 

This challenge is presently being engaged in the Pacific Northwest in the 

zOTKi of the WAI4I Program. WAMI is an acronym formed by using the first letters, 

of the states of Washington, Alaska, Montana and Idaho. It is the brainchild 
of three deans of the Medical School in Seattle, Dr. M. Roy Schwarz, presently 
Director of the WAMI Program; Dr. John N. Lein, Associate Dean for Continuing 
Education; and Dr. August P. Swanson, presently Director of Academic Affairs 
for the American Association of Medical Colleges. 

The idea nurtured in tlieir minds is a new concept in medical education. 
It is an experimental regionalization of parts of the University of Washington 
curriculum to the states of Alaska, Montana and Idaho. Fortunately the curric- 
ulum at tiie University of Washington as it now exists, and foreseable modifica- 
tion of it is amenable to the process of peripheralization. Roughly speaking 
the curriculum falls into two parts: basic science in the first two years and 
clinical science in the last tWo years. Parts of the curriculum in basic science 
can be taught at existing institutions in the states of Alaska, Montana and Idaho. 
Substantial clinical experience (in several primary care specialities) in the ^ 
third and fourth years can be provided in the rural areas of the four states 
by clinicians presently in practice in these areas. 
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EDUCATIONAL GOALS 
The educational goals of the WAMI Program are: 

1. To increase the number of students from the WAMI states accepted into 
Medical School. 

2. To utilize existing educational resources in the area7~thus^6Bviating 
the need for building new facilities. 

3. To provide educational opportunities and programs in non-metropolitan 
areas • 

4. To maintain quality educational programs in the region* 

5. To enhance the quality and availability of health care for all citizens 
of the WAMI area. 



PROGRAM DESCRIPTION 



To meet these goals the WAMI Program operates in two phases - the University 
Phase and the Clinical Phase* 

The Clinical Phase provides an opportunity for students in the third and 
fourth years of Medical School to enroll in clerkships remote from the Seatt/.e^ 
area^ Clerkships in Family Medicine, Obstetrics /Gynecology, internal Medicine, 
Psychiatry and Pediatrics have been or are being developed in small towns scat- 
tered throughout the region. Two students go to each clerkship on a six week 
rotation. Students may take more than one WAMI clerkship. The evaluation of 
this phase is so different from the evaluation of the University Phase, that 
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it merits a separate disctission at another time. This paper will concentrate 
on the University Phase* 

The University Phase of the WAMI Program provides students th2 opportunity 
of receiving the first quarter of their basic sciences at Universities in the 
WAMI area. "fhe^Jniversl-by^-of^l RsVa , Fai.rbnnV^, bogan witir^ students in the 
Fall of 1971. Washington State University, Pullman and the University of Idaho, 
Moscow, began in the Fall of 1972. In the Fall of 1973 Montana State University 
will be the foxxrth and final university to be involved. All four universities 
will operate concurrently/ each accomodating approximately 10 first year medical 
students. It is hoped that each university will ultimately provide one full 
year of the medical school basic science cxirriculum. 

The courses now offered by the participating universities are Biochemistry; 
Epidemiology; Physiology/Pharmacology; Anatomy /Histology; and Medicine, Health 
and Society, in addition a preceptorship with local practicing physicians is 
provided to introduce the student to the practice of medicine in the area. 



METHOD 

One of the goals of the University Phase is to provide the students at the 
WAMI Universities with an educational experience at least equivalent to that 
provided by the University of Washington Medical School. In the form of a ques- 
tion this goal is as follows: Is the educational experience at the peripheral 
universities equivalent to that provided at the University of Washington? 
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The challenge to the Office of Research in Medical Education is to find 
answers to that question. Charged with planning and conducting the evaluation 
of the WAMI Program in all its educational aspects the office has concentrated 
on obtaining answers to questions that relate to antecedents, process and out- 
comes of the WAMI University Phase, generating data in each -of -Uie-a£ea&-^aiid_. 
analysing the results. 



ANTECEDENTS 

Two groups of antecedent questions are questions about courses and faculty 
and questions about students* 

Do the outlying universities have available courses comparable to those 
available to the University of Washington students and do they have faculty 
necessary to teach them? These questions were answered on an administrative 
level. Some comparable courses existed and needed only minor modification to 
include content specific to medical education. Not having a medicine curriculum, 
courses in Biochemistry, Anatomy, Histology, Pharmacology, and Physiology did 
not have a medical orientation. Modifications of these courses provided this 
orientation. In some cases, new courses were developed and incorporated at the 
other universities. When additional faculty members were needed at the periph- 
eral sites, they were recruited and supported by the WAMI Program. 



students participating in the WAMI Program must first be admittsed to the 
University of Washington Medical School. They are then selected for the WAMI 
Program. Applicants to the University of Washington School of Medicine are in- 
formed of the WAMI Program and advised that they may either elect or be selected 
to pa rticipate. Native sons and daughters are encouraged to~ ^rLicipa L e dL Lli6lr 
home state institution. The important question is whether WAMI students differ 
from their classmates on college grade point averages, MCAT scores, or undergrad- 
uate major. 

PROCESS QUESTIONS 

Process questions deal with the delivery of the curricultjm content and the 
effect participating in the WAMI Program had on faculty and physician-preceptor 
time. 

The content of the curriculum had to be identified. To this end faculty 
members from all participating universities come together at a retreat to iden- 
tify content and to specify objectives for that content and to plan for the eval- 
uation of these objectives. The facility was grouped by course with the member 
from the University of Washington acting as chairman. These groups identified 
necessary and sufficient content for their course with varying degrees of suc- 
cess, which is discussed later. Prom this content outline - called common con- 
tent - they specified in a general way the objectives for each topic. Quecjtions 
for each topic objective were constructed and agreed upon and placed in a pool 
for each course. Prom this pool a portion of the final exam at each institution 
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would be constructed. This common test would optimally consist of 50 items 
chosen at random, but stratified as to course emphasis. Though substantially 
implemented, departures from this plan did occur and are noted in the section 
on results. It should be noted that identification of necessary and sufficient 
content did not preclude a teacher amplifying or expanding his course beyond 
the agreed upon common areast_j rhere wa s opportunity for each course to be indi- 
vidually developed and delivered. 

Evaluation of course process took the form of 

1. Weekly evaluation by students. 

2. Reports of visiting faculty from the University of Washington. 

3. Questionnaires administered pre and post to faculty, physician precep- 
tors, and students. 

4. Student structured interviews. 

Weekly evaluation by students provided valuable feedback to faculty on 
their objectives and the learning process. The visiting faculty program was 
implemented to decrease any sense of isolation students might have in being 
separated from classmates in Seattle. Each course had at least one visitor 
who gave a series of lectures in his subject. He reported his views of how 
the students were progressing and of how well the WAMI Program was being received 
by the various institutions. 

The faculty and physician/preceptors questionnaires are an attempt to as- 
certain how the WAMI Program is affecting the faculties time for research and 
professional development. For the physicians it attempted to find out how the 
program affected their practice both as to style and size. 
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Faculty were asked to identify problem areas in the curriculum and in the 
WAMI Program. They were also asked to indicate their perceptions of the chances 
for WAMI accomplishing its goals. 

Parts of the questionnaires were designed to give a general idea of what 
faculty and student attitudes were toward the WAMI Program both before the quarter 
began and after its completion. 

OUTCOME QUESTIONS 
Outcome Questions in the cognitive area are s 

1. Do the WAMI students master the necessary and sufficient content as 
well as their classmates in Seattle? 

2. Are the WAMI students as well prepared in basic sciences as the Univer- 
sity of Washington students? 

3. Do W7^ students do as well in subsequent course work as their clcissmates? 

The first question was answered by examining the results of the common testing 
program for evidence of non-comparability of content mastery. 

The second question is answered by analyzing scores on National Board Mini 
Tests at the beginning of the second year and by performance on the National 
Boards Part I examination at the end of the second yeax. 

Answers to the third question will be gleaned by monitoring each students* 
progress through medical school paying particular attention to his performance 
in University of Washington classes* 
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Outcome Questions in the affective area 

If students or faculty are al d > a program its chances for success 

are seriously hampered. Consequently it is important to have some measure of 
the attitudes of participants to the program. 

In a general way we asked what the attitudes of students were to the WAMI 
Program both prior to their experience and after its completion. 

This inquiry took the form of a simple questionnaire administered before 
they began their lectures and on their return to Seattle. They also engaged 
in a structured interview with University of Washington faculty on their return 
to Seattle. Important parameters are their enthusiasm for and support of .the 
program as well as their grasp of the goals of the WAMI Program and their per- 
ceptions of its chances for attaining these goals. 

RESULTS 

In this section we wish to emphasize the preliminary and tentative nature 
of the evaluation thus far. The ultimate questions about maldistribution and 
incr8;.:sed nximber of medical students can only be answered as data becomes avail- 
able. Questions about student's educational experience can only be partly answered 
now/ because several measures become available only as the students reach certain 
stages in their careers. For che purpose of this paper we are using the class 
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admitted in the Fall of 1972. Three of the WAMI states were involved in that 
quarter. [The class admitted in 1971 had only 9 students at one peripheral site 
and the data generated is not sxibstantially different from that presented here.] 

Antecedent Questions 

What are the student inputs to the program? 

From Tables I, II, and III you can see that the WAMI Groups do not differ 
from the entering class as a whole on mean CPA's and mean MCAT scores. Tests 
yielded no significant values for any of the conparisons. Their undergraduate 
majors are representative of the academic background of the class as a whole, 
with Chemistry, Biology and Zoology predominating. 

The analysis of student attitude toward the program, pre and post has not 
yet been completed. 
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TABLE I 



Mean Grade 



Total Entering Class 
Total WAMI Student Class 

Group 1 

Group 2 

Group 3 

U of W Class 



Point Averages 



Science Total N 

3.51 3.52 126 
3.53 3.50 30 
3.58 3.57 11 
3.57 3.49 9 
3.42 3.40 JO 

3.52 3.52 96 



TABLE II 



Mean MCAT Scores 



Verbal Quanlitative Genered. 
Ability Ability Information 



Total Entering Class 581 642 575 597 126 

Total WAMI Class 568 629 569 596 30 

Group 1 595 663 595 604 11 

Group 2 545 609 549 596 9 

Group 3 559 606 558 585 10 

U of W Class 584 647 577 597 96 
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TABLE III 
Undergraduate Majors 
Major U of W Class WAMI Class 



Liberal Arts 


11 


0 


Biochetnistry 


7 


1 




1 


1 


Biology 


34 


7 


Biological Science 


2 


0 


Chemistry 


19 


4 


Engineering 


4 


2 


Mathematics 


3 


0 


Microbiology 


3 


1 


Natural Science 


1 


1 


Physical Sciences 


6 


0 


Premedicine 


5. 


2 


Psychology 


7 


2 


Zoology 


15 


6 


No Major 


1 


1 


Other 


4 


0 


Pathology 


1 


0 


Science (other 
than above) 


2 


0 


TOTAL 


126 


30 
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Process Questions 

The students evaluated their course on a weekly basis. The information 
from these written evaluations is used to plan the courses for the following 
year. It is also valuable feedback to the faculty on their objectives and de- 
mands. On the basis of this information further modification of some courses 
was tmdertaken. The general consensus was that the courses were well delivered 
at all sites. 

Students at the peripheral sites may tend to feel isolated from their class- 
mates and from the University of Weishington Medical School. To cotmter this 
feeling of isolation a program of visiting faculty from the University of Wash- 
ington was initiated. Faculty from each course visited the institution and de- 
livered lectures in their areas. They interacted with students and faculty out- 
side of classes and reported no significant problems with course delivery or 
content. Some minor problems were reported and these lent support to students 
appraisal of course conduct. 

Faculty and faculty-preceptor questionnaires were administered before and 
after the quarter began. Tables IV and V are examples of this questionnaire 
with mean responses included, pre-scores (X) on top of the lines, post-scores (Y) 
below the lines. Few significant differences were found. 



15 



TABLE IV 
PRECEPTOR QUESTIONNAIRE 

Date 

Name 

Institution^ 

Position 

Major Teaching Responsibility^ 

WAMI Course (s) Taught 

How may Semesters or Quarters have you been involved in the WAMI 
Program? (Indiate "O" if this is the beginning of your 1st 
Semester or Quarter) 

Semester 

or 

Quarters 



Number of Preceptees you have: 



As you consider this year's involvement in the WAMI program, what are ^ 
your general reactions to that involvement? On each line below a continuum 
is defined by the pairs o£ words - one on either end. Place a check mark 
on a space on each continuum which corresponds most closely to your re- 
action to your involvement in the WAMI Program. 

1 2 3 4 5 
Enthusiastic x Hostile 

V 

Pessimistic 



Optimistic 



y- 



Apprehensive x Calm 

y 

Satisfied X Dissatisfied* 

Excited Z ^^""^^ 

Favorable x! Unfavorable 

y 

Additional comments: In particular, reasons for positive and/or negative 
shifts in your reactions since they were last solicited (not applicable tor 
initial reactions) 

How well informed do you feel you are on the broad objectives of the WAMI 
Program? 

Well informed x Ignorant 



Some of the WAMI goals are listed below. Based on your present knowledge 
and attitude, what is your opinion concerning the potential of the WAMI 
Program to achieve those goals? Indiate your opinion of the potential by 
making a check under the appropriate column where the column heading have 
the following meanings? 

VH - Potential for attaining the objective is Very High or almost certain 
H - Potential is High, but not certain 

M - There is a moderate chance of obtaining the objective 
L - There is a low chance of achieving the objective 
VL - The chance oF'the objective is Very Low or nearly non-existant 

Goals 

1. To achieve a better distribution 
of physicians in the region 

2. To increase the number of 
physicians in the region 

3. To reduce the necessity for 
duplicate medical education 
facilities in the region 

4. To increase the quality of 
physicians in the region 

5. To increase the number of x 

students from participating y 
states accepted into med school. 



VH H M L VL 

' k _ " 

y~ 

if — 

y 

X ___ 

y 

X ' 

y 



TABLE IV (continued) 
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What required quantitative changes in your professional responsibilities have 
you experienced (or anticipate experiencing) as a result of participation in 
the WAMI Program. Please feel free to comment after each item. Use the 
back of this sheet if you need more room. 



1. Patient contact hours 

2. Hours spent in hospital 

3. Time spent on rounds 

4. Study hours 

5. Teaching 

6. Continuing Education 

7. Number of Journals read 



Greatly Somewhat No Somewhat Greatly 

Increased Increased Change Reduced Reduced 



y 

x_ 

y 



^ X 

y 



. X 

y 



Other required changes in your responsibilities. (specify kind and amount 

of change) 



What qualitative changes have you experienced (or anticipate experiencing) 
as a result of participation in WAMI. Feel free to make an explanatory 
comment after each item. Use space on the back if necessary. 

Greatly Somewhat No Somehwat Greatly 

Increased Increased Change Reduced Reduced 

1. Quality of patient 
care 



2. Quality of patient 
records 



3. Quality of patient 
progress notes 



4. Level of competence 
exhibited 



5. Preparation for 
rounds 



6. Cooperation and stim- 
ulation of colleagues 

(in teaching effort) x 

y 
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In general, what is your judgement of the reaction of your colleagues who 
are not in the WAMI program to the participation of your clinic in the 
WAMI program (check by position on the continuum defined by the adjective 
pairs which is closest to your judgement). 

Non-participating colleagues reaction to WAMI Program 

Enthusiastic x Hostile 

y 

Optimistic x * Pessimistic* 

y 

Apprehensive x ^ Calm 

y 

Aware ^ x Unaware 

y 

Excited x Bored* 



Cooperative ' x ^ ^^^^^ Uncooperative 

y 

Additional Comments: 



In general, what is your judgement of the reaction o£ University Faculty 
in your community to the WAMI Program: 



Enthusiastic 

Optimistic 

Apprehensive 

Excited 

Aware 

Cooperative 



x^ 

X 

y' 



Reaction of Physicians to WAMI Program 

_ ^ ^ ^ Hostile 

_ ^ Pessimistic 

_ jc Calm 

y 

_ ^ ^ Bored 

_ ^ ^ ^ Unaware 

_ ^ ^ Uncooperative 



y 

X 



Additional Comments: 



TABLE IV (continued) 
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Please use this space to comment on problems you ant icipate- or have 
had as a result of your participation in the WAMI Program. 



ERIC 



TABLE V 
FACULTY QUESTIONNAIRE 



DATE 



NAME: INSTITUTION POSITION 

MAJOR TEACHING RESPONSIBILITY 

WAMI COURSE (COURSES) TAUGOT 



How many semesters or quarters have you been involved in the WAMI Program (in- 
dicate 0 if this is the beginning of your first semester or quarter) 



or 



semesters 
quarters . 
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As you consider this year's involvement in the WAMI Program, what are your 
general reactions to that involvement? On each line below a continuum is defined 
by the pairs of words - one on either end. Place a check mark on a space on each 
continuum which corresponds most closely to your reaction to your involvement 
in the WAMI 'Progr£un* 



Enthusiastic 

Optimistic 

Apprehensive 

Satisfied 

Excited 

Favorable 



X 

y 



X 

y 



X 

y 

X 

y 

X 

y 



X, 

y 



Hostile 

Pessimistic 

Calm 

Dissatisfied 
Bored 

Unfavorable 



Additional comments: In particular, reasons for positive and/or negative shifts 
in yoxxr reactions since they were last solicited (not applicable for initial 
reactions) : 



How well informed do you feel you are of the broad objectives of the WAMI Program? 
Well Informed ' x Ignorant* 

y 

Some of the WAMI goals eure listed below. Based on yoxir present knowledge and attitude 
what is your opinion concerning the potential of the WAMI Program to achieve those 
goals? Indicate your opinion of the potential by making a check under the appropriate 
column where the column headings have the following meanings: 



VH - potential for attaining the objective is Very High or almost certain 
H - potential is High, but not certain 

M - there is a moderate chance of obtcdning the objective 
L - there is a Low chance of achieving the objective 
VL - the chance of the objective is Very Low or nearly non-existant 

Goals 

VH H M L VL 

1. To achieve a better distribution of x ' 

physicians in the region y 

2. To increase the number of physiciams x 

in the region y — — ^— ^— 

3. To reduce the necessity for duplicating x 

medic£d education facilities in the region y 



TABLE V (continued) 



Goals, Continued 



4. To increase the quality of 

health care delivery in the y 
region by emphasizing 
continuing medical education 
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VH H M L VL 

X 



X 



5. To increase the number of „ 

students from participating y 
states accepted into medical 
school 

What required quantitative changes in your professional responsibilities have 
you experienced (or anticipate experiencing) as a result of participation in 
the WAMI Program. Please feel free to comment after each item. Use the back 
of this sheet if you need more room. 

Greatly Somewhat No Somewhat Greatly 
Increased Increased Change Reduced Reduced 

1. Classroom contact hours x 

y 

2. Total classroom student 

load V 



y 

3. Student advising x 

y 

4. Committee responsibilities x 

y 

5. Research effort 



6. Service to other depart- 
ments and to outside 

agencies x 

• y 

Other required changes in your responsibilities. (Specify kind and' amotmt of change) 



What qualitative changes have you experienced (or anticipate experiencing) as a 
result of participation in WAMI. Feel free to. nmke an explanatory comment after 
each item. Use space on the back if necessary. 

Greatly Somewhat No Somewhat Greatly 
Increased increased Change Reduced Reduced 

1. Attention to specification 

of instructional objectives x 



2. Sensitivity to student 
reaction 



3, Quality of evaluation pro- 
cedures 



4. Quality or organization of 

instructional objectives x 

y 



TABLE V (continued) 
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Greatly Somewhat No Somewhat Greatly 
Increased Increased Change Reduced Reduced 



5. Preparation in background 
material 



6. Amount of laboratory equip** 
ment 

7. Amount of visual and writt- 
en instructional material 



8. 



Cooperation and stimulation 
of colleagues (in teaching 
effort) 



y 
y 



Research: 

1. Time available for research 

2. Student assistance in re- 

search efforts 

3. Material (equipment) for 

research efforts 

4. Interest in research (yours) 

5. Cooperation and stimulation 3C 

of colleagues in research y 

effort 

In general, what is your judgment of the reaction of your colleagues who are 
not in the WAMI Program to the participation of your institution in the WAMI 
Program (check by position on the continuum defined by the adjective pairs which 
is closest to your judgment) • 



X 

y 



Enthusiastic 

Optimistic 

Apprehensive 

Aware 

Excited 

Cooperative 



_x 

y 

_x 

y 



Non-participating colleagues reaction to WAMI Program 

Hostile 

J Pessimistic 

jc Calm 

y 

25 Unaware* 

Bored 

Uncooperative 
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In general, what is your judgment of the reaction of physicians in your 
community to the WAMI Program: 



Reaction of Physicians to WAMI Program 
Enthusiastic x Hostile 

y 

optimistic x Pessimistic 

y 

Apprehensive jc Calm 

y 

Excited x Bored 

y 

Aware X Unaware * 



y 

CoojH.rativo x Uncooporativo 

y 

How would you judge the effects on your institution resulting from involvement 
in the WAMI Program? Please respond to the specific aspects listed below and 
then add any other comments you may have. Please check the blank which comes 
closest to your judgment. 

Greatly Somewhat No Somewhat Greatly 
Improved Improved Change Weakened Weakened 

Institutional Aspect 

1. Quality of instructional 
material: 

In the sciences x 

In the institution y — 

at large * x 

2. Quality of instructional 
methods: 

In the sciences x 

In the institution y 
at large v 



Quantity of instruction- 
al materials X 

y 

Intellectual stimula- 
tion . X 



y 



5. Interaction with col- 
leagues within insti- 
tution 

6. Interaction with col- 
leagues between insti- 
tution 

7. Interaction with individ- 
uals in the medical 

^ community 



TABLE V (continued) 
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I.istt'd bo low aro iujme iK)!j:;ible problem areas in the WAMI eooperativo effort. 
Please use your judgment to rate present or potential seriousness of the problem 
areas listed on the 5-point scale provided. Then, please add other problems 
and their degree of seriousness. Respond from the point of view of the course 
or courses you teach in the WAMI Program. 



Not Serious 



1. Mixture of WAMI and non- 
WAMI students in same 
course 



Extent of Problem 
2 3 4 



Serious 



2. The coordination of quarter 
and semester systems 

3. Time constraints in achieving_ 
course objectives 



y 

X 



4. Class size too large to 
achieve course objectives 



5. Discrepancy in course obj- 
ectives for WAMI and non- 
WAMI students 



^x ^ 

y 



6. Coordination among WAMI 
institutions 



.X . 

y 



7. Communication among faculty 
members in WAMI institutions 



.X 

y 



other (please specify) 



If you have judged certain problems to be serious, we would appreciate any 
proposed solution you may have. (You are not obligated to propose a solution 
here but we encourage you to do so.) Please identify the problem (from those 
above) before presenting your suggestion* 
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Outcome Questions 

Only the first outcome question can be studied at this time - did the WAMI 
students master the necessary and sufficient content as well as their classmates 
in Seattle? You will remember that five courses and a preceptorship were offered 
and each one necessitates some discussion here. 

For each course, except Medicine, Health and Society and the preceptorship, 
a common test was devised. By common we mean that the same set of items in each 
subject was given to all students, both at the peripheral sites and at the Uni- 
versity of Washington* 

This common test was planned at a faculty retreat in August, 1972. Facul- 
ties involved in the courses came together with their coiinterparts from each 
institution involved and identified necessary and sufficient content for each 
course. This necessary and sufficient content was labeled "common content*" 
Each institution was free to add to or amplify this content. Then the faculties, 
working together, constructed test items for the objectives of all common con- 
tent emphasis. The final set of items, consisting of items randomly selected 
from each of the strata, was sent to each institution for inclusion in their 
final exam. The plan was to have at least 50 items for each exam, and to add 

to the pool as the quarter or semester progressed. Some departures from this 

plan are described under each course heading below. 

Biochemistry 

* 

At the faculty retreat a pool of 58 items was constructed. Problems associ- 
ated with such a small pool were avoided when all faculties agreed that the 
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American Chemistry Association's cooperative test in Biochemistry, Form A was 
appropriate for the content and objectives of the course. 

The ACS test is an 80 item multiple choice exam with 4 choices for each 
question. The student's score is corrected for guessing by the usual formula. 
National norms are available for most of the ACS tests, but have not been com- 
piled for Form A of the Biochemistry test yet. An item analysis showed the 
test to have a reliability index of .80 (Kuder-Richardson 20) for this group 
with five items being consistently omitted. 

Figure 1.1 is frequency distribution of scores for this exam by all the 
students. 

Figure 2.1 is a curve from which a percentile equivalent Tor any given 
score can be read* 

Table 6.1 is the Analysis of Variance table for the WAMI group and the 
University of Washington group. An F-ratio of this low magnitude (.0111) indi- 
cates that there is no evidence that there is a difference between the means 
of the .groups represented by tliese students. 

Table 6.2 is the Analysis of Variance for the four groups - 3 WAMI groups 
and the University of Washington. 

Again the F-ratio (.2617) is significant. None of the group means differ 
significantly from any of the others. 

On the basis of this information it was concluded that all groups mastered 
the Biochemistry content at the same level and no further analysis was performed. 
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FREQUENCY 
14i 



Class E72 
N = 101 



PERCENT 
-14 



IS 32 36 40 44 48 52 56 60 64 68 
flC5 TEST IN BIOCHEMISTRY - HOME AND MflMI GROUP 



72 



(52.68*9.208) 101 VALUES 

FIGURE 1.1 - FREQUENCY DISTRIBUTION OF SCORES FOR 

ACS TEST IN BIOCHEMISTRY 




FIGURE 1.2 - PERCENTILE RATJKS OF SCORES FOR 

TEST IN BIOCHEMISTRY 



TABLE 6,1 

ANALYSIS OF VARIANCE FOR BIOCHEMISTRY - PERIPHERAL SITES AND U OF W 



U OF W 

Treatment Group 1 2 

Sample Size 30 71 

Mean 52.833 52.620 

Standard Deviation 8.611 9.571 



ANALYSIS OF VARIANCE 

SUM OF SQUARES DF MEAN SQUARE F RATIO 

Between Groups .9623 1 .9623 .0111 

Within Groups 8562.8991 99 86.4838 

TOTAL 8563.8614 100 
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TABLE 6.2 

ANALYSIS OF VARIANCE FOR BIOCHEMISTRY - PERIPHERAL SITES AND U OF W 



Treatment Group 
Sample size 
Mean 

Standard Deviation 



1 2 

9 10 

54.444 50.800 

4.558 10.086 



U OF W 
3 4 

11 71 

53.364 52.620 

10.003 9.571 



Between Groups 
Within Groups 



ANALYSIS OF VARIANCE 
SUM OF SQUARES DF MEAN SQUARE F RATIO 
68*7613 3 22.9204 .2617 

8495.1001 97 87.5784 



TOTAL 



8563.8614 



100 
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Anatomy 

A common test was compiled from over one hundred questions composed at the 
faculty retreat. The sample of 53 items was sent to each institution. Five 
items were eliminated resulting in a common test consisting of 48 items. This 
common test was incorporated into the final exam in Anatcxny at each institution. 
No criterion for passing was set as this was only a part of the final exam. 

An item analysis of this common test yielded a reliability index of only 
.48 (Kuder-Richardson 20). No items were consistently omitted. 

It is recognized that the low reliability will result in a statistical 
test with relatively low precision. It is expected that the reliability will 
be increased as the item pool is refined. 

Table 7.1 is the Analysis of Variance table for this exam for .WAMI and the 
University of Washington^ as can be seen a significant F-ratio (25.6863) is 
obtained r indicating that the means for the groups differ. 

Table 7.2 is the Analysis of Variance table for 4 groups, since the F-ratio 
is significant (9.496) r Scheffe's method of making comparisons among means was 
applied. 

Table 7.3 is the Matrix of Comparisons among means on the Anatomy common 
test. An asterisk indicates significant differences. Scheffe's method was 
applied to each pair of means. Means two^ three and fovr differed from mean 
one^ but not from each other. This provides evidence that the WAMI groups have 
mastered the common content for Anatomy better than the University of Washington. 
The source of this apparent superiority of the groups at the peripheral sites 



- 33 - 



has not been fully identified though there is some indication that it is due 
to the broader courses being offered at the peripheral sites. 



ERIC 
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FREQUENCY 
15, 



10 
9 

e 

7 
6 
5 
4 
3 
2 
1- 



Class E72 
N = 100 



PERCENT 
15 



J — L n 



>J 20 29 30 31 32 33 34 35 36 37 36 39 40 41 42 43 44 45 46 47 
flNflTOMY COMMON TEST SCORES - WflMI AND HOME 



(34.62,3.7731 100 VALUES 

FIGURE 2.1 - FREQUENCY DISTRIBUTION OF SCORES FOR 

ANATOMY COMMON TEST 1972 
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T ABLE 7,1 

ANALYSIS OF VARIANCE FOR ANATOMY COMMON TEST - PERIPHERAL SITE AND U OF W 

1972 



WAMI V OF y 
Treatment Group 1 2 
Sample Size 29 71 

Mean 37.310 33.521 

Standard Deviation 2,714 3,629 



Between Groups 
Within Groups 



ANALYSIS OF VARIANCE 
SUM OF SQUARES DF MEAN SQUARE F RATIO 
295 .6348 1 295 • 6348 25 ,6863 

1127.9252 98 11.5094 



TOTAL 



1423.5600 



99 
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TABLE 7.2 

ANALYSIS OF VARIANCE FOR ANATOMY COMMON TEST - PERIPHERAL SITE AND U OF W 

1972 



U OF W 

Treatment Group 12 3 4 

Sample Size 9 10 IQ 71 

Mean 36.778 36.400 38.700 33.521 

Standard Deviation 2.587 2.951 2.214 3.629 



Between Groups 
Within Groups 



ANALYSIS OF VARIANCE 
SUM OF SQUARES DP MEAN SQUARE F RATIO 
325.7861 • 3 108.5954 9.4966 
1097.7739 96 11.4351 



TOTAL 



1423.5600 



99 



ERIC 



. 3d - 

TABLE 7.3 

MATRIX OF COMPARISONS AMONG MEANS FOR ANATOMY COMMON TEST 

1972 



U OF W 







^1 


^2 


^3 


^4 


h 


33.52 




2.82 


3.28 


5.18* 




36.40 
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2.30 


h 


36.78 








1.92 


^4 


38.70 
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Phy s io logy /PharmacQ logy 

A pool of 120 items was constructed at the faculty retreat. From this pool 
40 items were randomly selected, stratified as to content emphasis and included 
on the final exam for this course at each site. 

An item analysis g>ve a reliability index of .62 (Kuder-Richardson 20). No 
items were consistently omitted. 

Table 8.1 and 8.2 are the Analysis of Variance tables for the WAMI groups 
and the University of Washington. A significant F-ratio (20.1926) was obtained 
indicating that the groups may differ. 

Table 8.3 is the Matrix of Comparisons means using Scheffe's method. An 
asterisk indicates a statistically significant difference. 

The interpretation is that the group 3 did not master the content as well 
as the University of Washington group. The University of Washington and groups 
1 and 2 mastered the common content equally well. 
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rREQUENCY 
16i 



Class E72 
N = 99 



PERCENT 
-jl6 

-15 

-14 

-13 

-12 

-11 

-10 

- 9 

- 8 

- 7 

- 6 

- 5 

- 4 
— 3 

■ 2 

■ 1 



II 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 
CONTROL SYSTEMS COMMON TEST SCORES - WflMI AND HOME 



133.23.3.731 99 VALUES 



FIGURE 3.1 - FREQUENCY DISTRIBUTION OF SCORES FOR 
PHYSIOLOGy/PHARMACOLOGY 1972 




FIGURE 3.2 - PERCENTILE RANKS OF SCORES FOR ' 

PHYSIOLOGY/PHARMACOLOGY COMMON TEST 1972 
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TABLE 8,1 

ANALYSIS OF V7VRIANCE FOR PHYSIOLOGY/PHARMACOLOGY COMMON TEST 

PERIPHERAL SITE AND U OF W 



Treatment Group 
Sample Size 
Mean 

Stcindard Deviation 



U OF W 
1 2 

27 72 

30.704 34.181 

4.084 3.155 



ANALYSIS OF VARIANCE 

SUM OF SQUARES DF MEAN SQUARE F RATIO 
Between Groups 237.3742 1 237.3742 20.1926 

Within Groups 1140.2824 97 11.7555 



TOTAL 



1377.6566 98 
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TABLE 8.2 

ANALYSIS OF VARIANCE FOR PHYSIOLOGY/PHARMACOLOGY COMMON TEST SCORES 
PERIPHERAL SITE AND U OF W 



Treatment Group 
Sample Size 
Mean 

Standard Deviation 



1 

9 

32.000 
5.339 



2 

7 

32.286 
2.059 



3 

11 
28.286 
3.171 



U OF W 
4 
72 
34.181 
3.155 



ANALYSIS OF VARIANCE 



Between Groups 
Within Groups 



SUM OF SQUARES DF 
317.0298 3 
1060.6268 95 



MEAN SQUARE 
105.6766 
11.1645 



F RATIO 
9.4654 



TOTAL 



1377.6566 



98 
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TABLE 

MATRIX OP COMPARISONS AMONG MEANS PGR PHYSIOLOGY/PHARMACOLOGY 
COMMON TEST SCORES 



U OP W 



U OP W 



^1 


28.64 




3.36 


3.65 


5.54* 




32.00 






.286 


2.18 


^3 


32.29 








1.89 


^4 


34.18 











Biostatistics and Epidemiology 

A pool of more than 150 items was constructed at the faculty retreat. From 
this pool of items the chairman of the course at the University of Washington 
constructed a 35 item common portion of the final exam. While these were mostly 
of a multiple choice type several were short answer questions. Ten of the mul- 
tiple choice items had sc^veral correct answers and students were expected to 
indicate all correct chc . An item analysis of this test has not yet been 
completed. For the pur- y i of comparing the groups the students' answer sheets 
were rescored with one \ ^ t for each totally correct answer. Some of the short 
answer questions were reconstructed into dichotomously scored items. The exam 
as reconstructed for comparison consisted of 40 items. 

Further problems with this test surfaced after its administration. Some 
instructors viewed some questions as ambiguous and reworded them for their site 
only. One site did not have an open book exam while all the others did. 

Frequency distributions of answers to each item are being prepared for the 
faculty to aid them in preparing the common test for next year. 
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1 



Class E72 
N = 102 



PERCENT 
14 



13 
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-11 
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8 
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3 
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' I 



14 15 16 17 18 19 ZO 21 22 23 24 25 26 27 28 29 30 31 32 
COnHON TEST IN BIOSTflTISTICS AND EPUOEHIOLOCY - MflHI AND HOriE 



(24.81.3.48) 102 VPLUES 



FIGURE 4,1 - FREQUENCY DISTKlBUTICN OF SCORES FOR BIOSTATISTICS 6 
EPIDEMIOLOGY COMMON TEST 1972 




FIGURE 4.2 - PERCENTILE RANKS OF SCORES FOR BIOSTATISTICS & 

EPIDEMIOLOGY COMMON TEST 1972 
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TABLE 9,1 

MEAN AND STANDARD DEVIATIONS FOR BlOSTATISTICS AND EPIDEMIOLOGY 
PERIPHERAL SITE AND U OF W 



U OF W 

Treatment Group 12 3 4 

Sample Size 9 9 11 73 

Mean 22.000 22.667 24.818 25.425 

Standard Deviation . 4.213 2.598 3.868 3.227 
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DISCUSSION AND CONCLUSION 

The evaluation of a program of the scale of WAMI is not completed tmtil 
all questions are addressed. What has been presented here is an attempt to 
answer only immediate questions about the University Phase of WAMI. Data 
representing one quarter of the program has been presented. After several 
quarters it will be more appropriate to attempt definitive answers to many of 
the questions raised. 

From data gathered during the first quarter, it seems appropriate to con- 
clude that, as far as outcome in the cognitive areas are concerned, the students 
at the peripheral sites mastered the necessary and sufficient content as well 
as their classmates at the University of Washington, it is also important to 
note that, from the comparisons performed there is no consistent trend for 
student performance at any site to be different from the others. Differences 
in individual courses may arise from a number of sources including teacher dif- 
ferences, time differences, and material emphasis. 

In making the comparison among the groups, Scheffe's method was tised. This 
is a conservative test and may miss some differences. But, in our judgment, 
it is the test that best accomodates the discrepancy in sample sizes that the 
situation gives rise to. 

The reliability of two of the common tests was not very high (.48 for Anatomy 
and .62 for Physio logy /Pharmacology ) . It is expected that these indices will 
be increased with further refinement of the test item pool. Plans for this 
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refinement are tander way. All faculty members involved met after the quarter 
was completed and discussed problems in course process and evaluation. A com- 
mitment has been made to increase the number of items in all courses (except 
Biochemistry/ which doesn't need refinement) including Biostatistics and Epidem- 
iology. If these commitments are met more precise instruments will be available 
for future evaluation. 

Student attitude for this quarter has not yet been fully analysed. Data 
from informal conversations, preliminary survey of returned questionnaires/ 
and completed structured interviews/ suggest that students are excremely en- 
thusiasitc and supportive of the progrcun. 

Faculty and preceptor attitude/ measured pre, and post shows very little 
change. Their initial enthusiasm has not been dampened/ nor were their expec- 
tations at the beginning unrealistic. They view the WAMI Program as enhancing 
their institution with much interest expressed by their non-participating colleagues* 

The WAMI Program envisions a year of the basic science curriculum at the 
peripheral sites in the near future. Plans are already being formulated to 
expand to one year. The evaluation will be basically the same except there 
will be approximately fiftaen courses instead of five. Summative evaluation 
in the cognitive area may take the form of a comprehensive test administered 
to all students at the end of the first year or beginning of the second. The 
National Board Mini-test is presently administered at the beginning of the sec- 
ond year and may be appropriate for summative evaluation. 
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Should the WAMI Program prove to be a viable and desireable way to educate 
mebical students, the participating WAMI states (Alaska, Montana, Idaho) would 
be expected to assume financial responsibility for educating their citizens* 
Because the WAMI Program may eventually attract more students from the states 
than is now possible, and may result in a more satisfactory distribution of 
physicians than currently exists it is hoped that the quality and availability 
of health care for all citizens of the WAMI area would be improved. 



